منابع مشابه
Extrinsic Evaluation of Automatic Metrics for Summarization
This paper describes extrinsic-task evaluation of summarization. We show that it is possible to save time using summaries for relevance assessment without adversely impacting the degree of accuracy that would be possible with full documents. In addition, we demonstrate that the extrinsic task we have selected exhibits a high degree of interannotator agreement, i.e., consistent relevance decisio...
متن کاملA Methodology for Extrinsic Evaluation of Text Summarization: Does ROUGE Correlate?
This paper demonstrates the usefulness of summaries in an extrinsic task of relevance judgment based on a new method for measuring agreement, Relevance-Prediction, which compares subjects’ judgments on summaries with their own judgments on full text documents. We demonstrate that this is a more reliable measure than previous measures based on gold standards. Because it it more reliable, we are ...
متن کاملSummarization Evaluation : Correlating Human Performance on an Extrinsic Task with Automatic Intrinsic Metrics
Title of dissertation: Text Summarization Evaluation: Correlating Human Performance on an Extrinsic Task with Automatic Intrinsic Metrics Stacy F. Hobson Doctor of Philosophy, 2007 Dissertation directed by: Professor Bonnie J. Dorr Department of Computer Science Text summarization evaluation is the process of assessing the quality of an individual summary produced by human or automatic methods....
متن کاملIntrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization
This paper demonstrates the usefulness of summaries in an extrinsic task of relevance judgment based on a new method for measuring agreement, Relevance-Prediction, which compares subjects’ judgments on summaries with their own judgments on full text documents. We demonstrate that, because this measure is more reliable than previous gold-standard measures, we are able to make stronger statistica...
متن کاملDetection of Topic and its Extrinsic Evaluation Through Multi-Document Summarization
This paper presents a method for detecting words related to a topic (we call them topic words) over time in the stream of documents. Topic words are widely distributed in the stream of documents, and sometimes they frequently appear in the documents, and sometimes not. We propose a method to reinforce topic words with low frequencies by collecting documents from the corpus, and applied Latent D...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: ACM Transactions on Speech and Language Processing
سال: 2009
ISSN: 1550-4875,1550-4883
DOI: 10.1145/1596517.1596518